{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CH5 数据预处理（淘米洗菜）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "无论是业务数据还是从外部获取的数据，均会有数据不规整的情况（缺失数据、重复数据、异常数据）<br>\n",
    "数据分析中有一句话：Garbage in, Garbage out<br>\n",
    "也就是说如果在分析或建模之前没有将数据处理干净，输出得到的模型与分析报告必然也能是垃圾<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.1 缺失值处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1.1缺失值查看"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在Python中，info()方法除了看字段类型之外，还可以观察是否有Missing Value的情况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 4 entries, 0 to 3\n",
      "Data columns (total 4 columns):\n",
      "编号      4 non-null object\n",
      "年龄      4 non-null int64\n",
      "性别      3 non-null object\n",
      "注册时间    4 non-null object\n",
      "dtypes: int64(1), object(3)\n",
      "memory usage: 208.0+ bytes\n"
     ]
    }
   ],
   "source": [
    "data_dict = {\"编号\":[\"A1\",\"A2\",\"A3\",\"A4\"],\"年龄\":[54,16,47,41],\n",
    "        \"性别\":[\"男\",np.nan,\"女\",\"男\"],\n",
    "        \"注册时间\":[\"2018-8-8\",\"2018-8-9\",\"2018-8-10\",\"2018-8-11\"]}\n",
    "df = pd.DataFrame(data_dict)\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "性别字段是3 non-null object，表示性别这一列有3个非null值，而其他列有4个非null值，说明性别这一列只有1个null<br>\n",
    "当然，也可以直接用isnull()方法，来判断"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>编号</th>\n",
       "      <th>年龄</th>\n",
       "      <th>性别</th>\n",
       "      <th>注册时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      编号     年龄     性别   注册时间\n",
       "0  False  False  False  False\n",
       "1  False  False   True  False\n",
       "2  False  False  False  False\n",
       "3  False  False  False  False"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_dict = {\"编号\":[\"A1\",\"A2\",\"A3\",\"A4\"],\"年龄\":[54,16,47,41],\n",
    "        \"性别\":[\"男\",np.nan,\"女\",\"男\"],\n",
    "        \"注册时间\":[\"2018-8-8\",\"2018-8-9\",\"2018-8-10\",\"2018-8-11\"]}\n",
    "df = pd.DataFrame(data_dict)\n",
    "df.isnull()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了方便期间，可以直接对每列的True进行求和"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "编号      0\n",
       "年龄      0\n",
       "性别      1\n",
       "注册时间    0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum(axis = 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1.2缺失值删除"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "删除缺失值可以用dropna()方法，默认是删除有缺失值的行（只要有缺失值的就会进行删除）<br>\n",
    "可以用参数how = \"all\"，只删除整行均为缺失值的行<br>\n",
    "不过为了保险期间，在drop之前可以查看是哪些行存在Missing Value <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_dict = {\"编号\":[\"A1\",\"A2\",np.nan,\"A4\"],\"年龄\":[54,16,np.nan,41],\n",
    "        \"性别\":[\"男\",np.nan,np.nan,\"男\"],\n",
    "        \"注册时间\":[\"2018-8-8\",\"2018-8-9\",np.nan,\"2018-8-11\"]}\n",
    "df = pd.DataFrame(data_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>编号</th>\n",
       "      <th>年龄</th>\n",
       "      <th>性别</th>\n",
       "      <th>注册时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A2</td>\n",
       "      <td>16.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2018-8-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    编号    年龄   性别      注册时间\n",
       "1   A2  16.0  NaN  2018-8-9\n",
       "2  NaN   NaN  NaN       NaN"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#查看性别为缺失值的行\n",
    "df[df[\"性别\"].isnull()]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>编号</th>\n",
       "      <th>年龄</th>\n",
       "      <th>性别</th>\n",
       "      <th>注册时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A1</td>\n",
       "      <td>54.0</td>\n",
       "      <td>男</td>\n",
       "      <td>2018-8-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A4</td>\n",
       "      <td>41.0</td>\n",
       "      <td>男</td>\n",
       "      <td>2018-8-11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   编号    年龄 性别       注册时间\n",
       "0  A1  54.0  男   2018-8-8\n",
       "3  A4  41.0  男  2018-8-11"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dropna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>编号</th>\n",
       "      <th>年龄</th>\n",
       "      <th>性别</th>\n",
       "      <th>注册时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A1</td>\n",
       "      <td>54.0</td>\n",
       "      <td>男</td>\n",
       "      <td>2018-8-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A2</td>\n",
       "      <td>16.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2018-8-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A4</td>\n",
       "      <td>41.0</td>\n",
       "      <td>男</td>\n",
       "      <td>2018-8-11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   编号    年龄   性别       注册时间\n",
       "0  A1  54.0    男   2018-8-8\n",
       "1  A2  16.0  NaN   2018-8-9\n",
       "3  A4  41.0    男  2018-8-11"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dropna(how = \"all\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.1.3缺失值填充"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因为数据是宝贵的，只要缺失比例控制在一定的范围内就可以接受<br>\n",
    "在实际工作中，比起直接删除，首选应该是进行填充<br>\n",
    "填充的方法有：<br>\n",
    "1.直接用0<br>\n",
    "2.数值型用众数、平均数、中位数填充<br>\n",
    "&nbsp;&nbsp;文本型用众数<br>\n",
    "3.直接将有缺失值的行导出，交给业务部的同事手工确认后再次写入<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        男\n",
       "1    Other\n",
       "2        女\n",
       "3        男\n",
       "Name: 性别, dtype: object"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_dict = {\"编号\":[\"A1\",\"A2\",\"A3\",\"A4\"],\"年龄\":[54,16,np.nan,41],\n",
    "        \"性别\":[\"男\",np.nan,\"女\",\"男\"],\n",
    "        \"注册时间\":[\"2018-8-8\",\"2018-8-9\",\"2018-8-10\",\"2018-8-11\"]}\n",
    "df = pd.DataFrame(data_dict)\n",
    "df[\"性别\"].fillna(value=\"Other\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>编号</th>\n",
       "      <th>年龄</th>\n",
       "      <th>性别</th>\n",
       "      <th>注册时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A1</td>\n",
       "      <td>54.0</td>\n",
       "      <td>男</td>\n",
       "      <td>2018-8-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A2</td>\n",
       "      <td>16.0</td>\n",
       "      <td>男</td>\n",
       "      <td>2018-8-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A3</td>\n",
       "      <td>37.0</td>\n",
       "      <td>女</td>\n",
       "      <td>2018-8-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A4</td>\n",
       "      <td>41.0</td>\n",
       "      <td>男</td>\n",
       "      <td>2018-8-11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   编号    年龄 性别       注册时间\n",
       "0  A1  54.0  男   2018-8-8\n",
       "1  A2  16.0  男   2018-8-9\n",
       "2  A3  37.0  女  2018-8-10\n",
       "3  A4  41.0  男  2018-8-11"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_dict = {\"编号\":[\"A1\",\"A2\",\"A3\",\"A4\"],\"年龄\":[54,16,np.nan,41],\n",
    "        \"性别\":[\"男\",np.nan,\"女\",\"男\"],\n",
    "        \"注册时间\":[\"2018-8-8\",\"2018-8-9\",\"2018-8-10\",\"2018-8-11\"]}\n",
    "df = pd.DataFrame(data_dict)\n",
    "df.fillna({\"性别\":\"男\",\"年龄\":df[\"年龄\"].mean()})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2 重复值处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "与缺失值一样，重复的数据，会给数据统计与建模工作造成困扰<br>\n",
    "重复值处理也同样很有必要"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_dict = {\"订单编号\":[\"A1\",\"A2\",\"A3\",\"A3\",\"A4\",\"A5\"],\n",
    "             \"客户姓名\":[\"张通\",\"李谷\",\"孙凤\",\"孙凤\",\"赵恒\",\"赵恒\"],\n",
    "             \"唯一识别码\":[101,102,103,103,104,104],\n",
    "             \"成交时间\":[\"2018-8-8\",\"2018-8-9\",\"2018-8-10\",\"2018-8-10\",\"2018-8-11\",\"2018-8-12\"]}\n",
    "df = pd.DataFrame(data_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2.1重复值查看"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在Python中，使用duplicated()判断是否为重复值，默认判断标准为整行，也可以加入参数subset，来指定同时重复才显示的行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1    False\n",
       "2    False\n",
       "3     True\n",
       "4    False\n",
       "5    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.duplicated(subset = ['唯一识别码','成交时间'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>订单编号</th>\n",
       "      <th>客户姓名</th>\n",
       "      <th>唯一识别码</th>\n",
       "      <th>成交时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A3</td>\n",
       "      <td>孙凤</td>\n",
       "      <td>103</td>\n",
       "      <td>2018-8-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A3</td>\n",
       "      <td>孙凤</td>\n",
       "      <td>103</td>\n",
       "      <td>2018-8-10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  订单编号 客户姓名  唯一识别码       成交时间\n",
       "2   A3   孙凤    103  2018-8-10\n",
       "3   A3   孙凤    103  2018-8-10"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[df.duplicated(keep=False, subset = ['唯一识别码','成交时间']), :]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.2.2重复值删除"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用drop_duplicates()判断是否为重复值，默认判断标准也为整行，也可以加入参数subset，来指定同时重复才删除的行<br>\n",
    "还有参数keep（保留哪一行），默认为first，即保留第一行，也可以传入last（保留最后一行）或者False（全部删除）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_dict = {\"订单编号\":[\"A1\",\"A2\",\"A3\",\"A3\",\"A4\",\"A5\"],\n",
    "             \"客户姓名\":[\"张通\",\"李谷\",\"孙凤\",\"孙凤\",\"赵恒\",\"赵恒\"],\n",
    "             \"唯一识别码\":[101,102,103,103,104,104],\n",
    "             \"成交时间\":[\"2018-8-8\",\"2018-8-9\",\"2018-8-10\",\"2018-8-10\",\"2018-8-11\",\"2018-8-12\"]}\n",
    "df = pd.DataFrame(data_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>订单编号</th>\n",
       "      <th>客户姓名</th>\n",
       "      <th>唯一识别码</th>\n",
       "      <th>成交时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A1</td>\n",
       "      <td>张通</td>\n",
       "      <td>101</td>\n",
       "      <td>2018-8-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A2</td>\n",
       "      <td>李谷</td>\n",
       "      <td>102</td>\n",
       "      <td>2018-8-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A3</td>\n",
       "      <td>孙凤</td>\n",
       "      <td>103</td>\n",
       "      <td>2018-8-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A4</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>A5</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  订单编号 客户姓名  唯一识别码       成交时间\n",
       "0   A1   张通    101   2018-8-8\n",
       "1   A2   李谷    102   2018-8-9\n",
       "2   A3   孙凤    103  2018-8-10\n",
       "4   A4   赵恒    104  2018-8-11\n",
       "5   A5   赵恒    104  2018-8-12"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.drop_duplicates()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>订单编号</th>\n",
       "      <th>客户姓名</th>\n",
       "      <th>唯一识别码</th>\n",
       "      <th>成交时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A1</td>\n",
       "      <td>张通</td>\n",
       "      <td>101</td>\n",
       "      <td>2018-8-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A2</td>\n",
       "      <td>李谷</td>\n",
       "      <td>102</td>\n",
       "      <td>2018-8-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A3</td>\n",
       "      <td>孙凤</td>\n",
       "      <td>103</td>\n",
       "      <td>2018-8-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A4</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>A5</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  订单编号 客户姓名  唯一识别码       成交时间\n",
       "0   A1   张通    101   2018-8-8\n",
       "1   A2   李谷    102   2018-8-9\n",
       "2   A3   孙凤    103  2018-8-10\n",
       "4   A4   赵恒    104  2018-8-11\n",
       "5   A5   赵恒    104  2018-8-12"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.drop_duplicates(subset = ['唯一识别码','成交时间'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>订单编号</th>\n",
       "      <th>客户姓名</th>\n",
       "      <th>唯一识别码</th>\n",
       "      <th>成交时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A1</td>\n",
       "      <td>张通</td>\n",
       "      <td>101</td>\n",
       "      <td>2018-8-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A2</td>\n",
       "      <td>李谷</td>\n",
       "      <td>102</td>\n",
       "      <td>2018-8-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A4</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>A5</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  订单编号 客户姓名  唯一识别码       成交时间\n",
       "0   A1   张通    101   2018-8-8\n",
       "1   A2   李谷    102   2018-8-9\n",
       "4   A4   赵恒    104  2018-8-11\n",
       "5   A5   赵恒    104  2018-8-12"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.drop_duplicates(subset = ['唯一识别码','成交时间'], keep = False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.3 异常值检测与处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.3.1异常值查看"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "异常值的判断方法有如下几种方式：<br>\n",
    "1.业务部同事人为定义，超过或低于某个值边算作异常值<br>\n",
    "2.绘制盒须图，大于上边缘(Q3+1.5×IQR)或小于下边缘（Q1-1.5×IQR）<br>\n",
    "3.若数据符合正态分布，与平均值超过3倍σ（标准差）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.3.2异常值处理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "异常值的处理方法有如下几种方式：<br>\n",
    "1.最常见的方式就是直接删除<br>\n",
    "2.当作缺失值进行填充<br>\n",
    "3.把异常值作为特殊情况，研究出现的原因<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.4 数据类型转化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pandas共有如下几种类型，<br>\n",
    "int：整数<br>\n",
    "float：浮点数<br> \n",
    "object：Python对象类型<br>\n",
    "string_：字符串类型<br>\n",
    "unicode_：固定长度的unicode类型<br>\n",
    "datetime64[ns]： 表示时间格式"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "除了之前提到的info()方法，还可以用dtype方法单独查看每个字段"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_dict = {\"订单编号\":[\"A1\",\"A2\",\"A3\",\"A3\",\"A4\",\"A5\"],\n",
    "             \"客户姓名\":[\"张通\",\"李谷\",\"孙凤\",\"孙凤\",\"赵恒\",\"赵恒\"],\n",
    "             \"唯一识别码\":[101,102,103,103,104,104],\n",
    "             \"成交时间\":[\"2018-8-8\",\"2018-8-9\",\"2018-8-10\",\"2018-8-10\",\"2018-8-11\",\"2018-8-12\"]}\n",
    "df = pd.DataFrame(data_dict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('O')"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"订单编号\"].dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('int64')"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"唯一识别码\"].dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python中，用astype()方法来对数据类型进行转换"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    101.0\n",
       "1    102.0\n",
       "2    103.0\n",
       "3    103.0\n",
       "4    104.0\n",
       "5    104.0\n",
       "Name: 唯一识别码, dtype: float64"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"唯一识别码\"].astype(\"float64\")\n",
    "# df[\"唯一识别码\"].dtype"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.5 索引设置"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "索引是查找数据的重要依据，目的是为了便于查找数据<br>\n",
    "例如，买完菜回家，把蔬菜水果放在零度间，肉类放在冷冻室<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame([ [\"A1\",\"张通\",101,\"2018-8-8\"],\n",
    "               [\"A2\",\"李谷\",102,\"2018-8-9\"],\n",
    "               [\"A3\",\"孙凤\",103,\"2018-8-10\"],\n",
    "               [\"A4\",\"赵恒\",104,\"2018-8-11\"],\n",
    "               [\"A5\",\"赵恒\",104 ,\"2018-8-12\"]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.5.1为无索引表添加索引"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用columns方法添加列索引，index方法添加行索引"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>订单编号</th>\n",
       "      <th>客户姓名</th>\n",
       "      <th>唯一识别码</th>\n",
       "      <th>成交时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A1</td>\n",
       "      <td>张通</td>\n",
       "      <td>101</td>\n",
       "      <td>2018-8-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A2</td>\n",
       "      <td>李谷</td>\n",
       "      <td>102</td>\n",
       "      <td>2018-8-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A3</td>\n",
       "      <td>孙凤</td>\n",
       "      <td>103</td>\n",
       "      <td>2018-8-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A4</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>A5</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  订单编号 客户姓名  唯一识别码       成交时间\n",
       "1   A1   张通    101   2018-8-8\n",
       "2   A2   李谷    102   2018-8-9\n",
       "3   A3   孙凤    103  2018-8-10\n",
       "4   A4   赵恒    104  2018-8-11\n",
       "5   A5   赵恒    104  2018-8-12"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns = [\"订单编号\",\"客户姓名\",\"唯一识别码\",\"成交时间\"]\n",
    "df.index = [1,2,3,4,5]\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.5.2重新设置索引"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>订单编号</th>\n",
       "      <th>客户姓名</th>\n",
       "      <th>唯一识别码</th>\n",
       "      <th>成交时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A1</td>\n",
       "      <td>张通</td>\n",
       "      <td>101</td>\n",
       "      <td>2018-8-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A2</td>\n",
       "      <td>李谷</td>\n",
       "      <td>102</td>\n",
       "      <td>2018-8-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A3</td>\n",
       "      <td>孙凤</td>\n",
       "      <td>103</td>\n",
       "      <td>2018-8-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A4</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>A5</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  订单编号 客户姓名  唯一识别码       成交时间\n",
       "1   A1   张通    101   2018-8-8\n",
       "2   A2   李谷    102   2018-8-9\n",
       "3   A3   孙凤    103  2018-8-10\n",
       "4   A4   赵恒    104  2018-8-11\n",
       "5   A5   赵恒    104  2018-8-12"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_dict = {\"订单编号\":[\"A1\",\"A2\",\"A3\",\"A4\",\"A5\"],\n",
    "             \"客户姓名\":[\"张通\",\"李谷\",\"孙凤\",\"赵恒\",\"赵恒\"],\n",
    "             \"唯一识别码\":[101,102,103,104,104],\n",
    "             \"成交时间\":[\"2018-8-8\",\"2018-8-9\",\"2018-8-10\",\"2018-8-11\",\"2018-8-12\"]}\n",
    "df = pd.DataFrame(data_dict)\n",
    "df.index = [1,2,3,4,5]\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "重新设置索引，一般是指重新设置行索引，虽然有些表已经自带了行索引，但这不是我们想要的，需要手动再制定列"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>客户姓名</th>\n",
       "      <th>唯一识别码</th>\n",
       "      <th>成交时间</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>订单编号</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>A1</th>\n",
       "      <td>张通</td>\n",
       "      <td>101</td>\n",
       "      <td>2018-8-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>A2</th>\n",
       "      <td>李谷</td>\n",
       "      <td>102</td>\n",
       "      <td>2018-8-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>A3</th>\n",
       "      <td>孙凤</td>\n",
       "      <td>103</td>\n",
       "      <td>2018-8-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>A4</th>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>A5</th>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     客户姓名  唯一识别码       成交时间\n",
       "订单编号                       \n",
       "A1     张通    101   2018-8-8\n",
       "A2     李谷    102   2018-8-9\n",
       "A3     孙凤    103  2018-8-10\n",
       "A4     赵恒    104  2018-8-11\n",
       "A5     赵恒    104  2018-8-12"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.set_index(\"订单编号\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.5.3重命名索引"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "重命名索引就是指对现有的索引名进行修改，用字典进行传入，一般针对列会用的多一些"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>订单编号</th>\n",
       "      <th>客户姓名</th>\n",
       "      <th>唯一识别码</th>\n",
       "      <th>成交时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A1</td>\n",
       "      <td>张通</td>\n",
       "      <td>101</td>\n",
       "      <td>2018-8-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>A2</td>\n",
       "      <td>李谷</td>\n",
       "      <td>102</td>\n",
       "      <td>2018-8-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>A3</td>\n",
       "      <td>孙凤</td>\n",
       "      <td>103</td>\n",
       "      <td>2018-8-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A4</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>A5</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  订单编号 客户姓名  唯一识别码       成交时间\n",
       "1   A1   张通    101   2018-8-8\n",
       "2   A2   李谷    102   2018-8-9\n",
       "3   A3   孙凤    103  2018-8-10\n",
       "4   A4   赵恒    104  2018-8-11\n",
       "5   A5   赵恒    104  2018-8-12"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_dict = {\"订单编号\":[\"A1\",\"A2\",\"A3\",\"A4\",\"A5\"],\n",
    "             \"客户姓名\":[\"张通\",\"李谷\",\"孙凤\",\"赵恒\",\"赵恒\"],\n",
    "             \"唯一识别码\":[101,102,103,104,104],\n",
    "             \"成交时间\":[\"2018-8-8\",\"2018-8-9\",\"2018-8-10\",\"2018-8-11\",\"2018-8-12\"]}\n",
    "df = pd.DataFrame(data_dict)\n",
    "df.index = [1,2,3,4,5]\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>新订单编号</th>\n",
       "      <th>新客户姓名</th>\n",
       "      <th>唯一识别码</th>\n",
       "      <th>成交时间</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>一</th>\n",
       "      <td>A1</td>\n",
       "      <td>张通</td>\n",
       "      <td>101</td>\n",
       "      <td>2018-8-8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>二</th>\n",
       "      <td>A2</td>\n",
       "      <td>李谷</td>\n",
       "      <td>102</td>\n",
       "      <td>2018-8-9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>三</th>\n",
       "      <td>A3</td>\n",
       "      <td>孙凤</td>\n",
       "      <td>103</td>\n",
       "      <td>2018-8-10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>A4</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>A5</td>\n",
       "      <td>赵恒</td>\n",
       "      <td>104</td>\n",
       "      <td>2018-8-12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  新订单编号 新客户姓名  唯一识别码       成交时间\n",
       "一    A1    张通    101   2018-8-8\n",
       "二    A2    李谷    102   2018-8-9\n",
       "三    A3    孙凤    103  2018-8-10\n",
       "4    A4    赵恒    104  2018-8-11\n",
       "5    A5    赵恒    104  2018-8-12"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.rename(columns = {\"订单编号\":\"新订单编号\",\"客户姓名\":\"新客户姓名\"}, index = {1:\"一\",2:\"二\",3:\"三\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.5.4重置索引"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同样的，如果我们对原有索引不满意时，可以用reset_index()方法直接进行充值，多用于数据分组与数据透视表中<br>\n",
    "将所列全都转为行，行索引转为列，并用从0开始的默认索引,在设置索引时，还有一个参数drop，默认为True，即作为索引的列，会直接在原表进行删除"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>C1</th>\n",
       "      <th>C2</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Z1</th>\n",
       "      <th>Z2</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">A</th>\n",
       "      <th>a</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">B</th>\n",
       "      <th>b</th>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       C1  C2\n",
       "Z1 Z2        \n",
       "A  a    1   2\n",
       "   a    3   4\n",
       "B  b    5   6\n",
       "   b    7   8"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_dict = {\"Z1\":[\"A\",\"A\",\"B\",\"B\"],\n",
    "             \"Z2\":[\"a\",\"a\",\"b\",\"b\"],\n",
    "             \"C1\":[1,3,5,7],\n",
    "             \"C2\":[2,4,6,8]}\n",
    "df = pd.DataFrame(data_dict)\n",
    "df = df.set_index(keys=['Z1','Z2'])\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Z1</th>\n",
       "      <th>Z2</th>\n",
       "      <th>C1</th>\n",
       "      <th>C2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A</td>\n",
       "      <td>a</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>A</td>\n",
       "      <td>a</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>B</td>\n",
       "      <td>b</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>B</td>\n",
       "      <td>b</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Z1 Z2  C1  C2\n",
       "0  A  a   1   2\n",
       "1  A  a   3   4\n",
       "2  B  b   5   6\n",
       "3  B  b   7   8"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Z1</th>\n",
       "      <th>C1</th>\n",
       "      <th>C2</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Z2</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>A</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>a</th>\n",
       "      <td>A</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>B</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>b</th>\n",
       "      <td>B</td>\n",
       "      <td>7</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Z1  C1  C2\n",
       "Z2           \n",
       "a   A   1   2\n",
       "a   A   3   4\n",
       "b   B   5   6\n",
       "b   B   7   8"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.reset_index(level = 0)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
